Data Capture System for Texture and Geometry Acquisition

ABSTRACT

Examples of the present disclosure describe systems and methods for capturing data to acquire indoor and outdoor geometry. In aspects, a data capture system may be configured to acquire texture data, geometry data, navigation data and/or orientation data to support geolocation and georeferencing within indoor and outdoor environments. The data capture system may further be configured to acquire seamless texture data from a 360° horizontal and vertical perspective to support panoramic video and images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application and claims priority toU.S. patent application Ser. No. 14/812,565, filed Jul. 29, 2015, andentitled “Data Capture System for Texture and Geometry Acquisition,”which application is incorporated herein by reference in its entirety.

BACKGROUND

Mobile mapping is the process of collecting geospatial data from amobile device fitted with a one or more remote sensing systems. TheGlobal Navigation Satellite System (GNSS), for example, is a satellitenavigation system that provides location and mapping information on ornear the surface of the earth. Typically, because GNSS requires a lineof sight to a plurality of satellites, GNSS will not function throughobstructions (e.g., indoors). For such reasons, alternate portablesystems must be used to provide location and mapping information formany urban areas and indoor environments.

It is with respect to these and other general considerations that theaspects disclosed herein have been made. Also, although relativelyspecific problems may be discussed, it should be understood that theexamples should not be limited to solving the specific problemsidentified in the background or elsewhere in this disclosure.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription section. This summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used as an aid in determining the scope of the claimedsubject matter.

Examples of the present disclosure describe systems and methods forcapturing data to acquire indoor and outdoor texture and geometry. Inaspects, a data capture system may be configured to acquire texturedata, geometry data, navigation data and/or orientation data to supportgeolocation and georeferencing within indoor and outdoor environments.The data capture system may further be configured to acquire seamlesstexture data from a 360° horizontal and vertical perspective to supportpanoramic video and images.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Additionalaspects, features, and/or advantages of examples will be set forth inpart in the description which follows and, in part, will be apparentfrom the description, or may be learned by practice of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive examples are described with reference tothe following figures.

FIG. 1 illustrates an overview of an example system for capturing datato acquire indoor and outdoor texture and geometry as described herein.

FIG. 2A illustrates an exemplary data capture system as describedherein.

FIG. 2B illustrates an exemplary data capture system mounted to a rigidframe as described herein.

FIG. 3A illustrates an exemplary panoramic head as described herein.

FIG. 3B illustrates an exemplary active and passive cooling system ofthe panoramic head of FIG. 3A as described herein.

FIG. 4 illustrates an example method of mapping indoor and outdoortexture and geometry as described herein.

FIG. 5 is a block diagram illustrating an example of a computing devicewith which aspects of the present disclosure may be practiced.

FIGS. 6A and 6B are simplified block diagrams of a mobile computingdevice with which aspects of the present disclosure may be practiced.

FIG. 7 is a simplified block diagram of a distributed computing systemin which aspects of the present disclosure may be practiced.

DETAILED DESCRIPTION

Various aspects of the disclosure are described more fully below withreference to the accompanying drawings, which form a part hereof, andwhich show specific exemplary aspects. However, different aspects of thedisclosure may be implemented in many different forms and should not beconstrued as limited to the aspects set forth herein; rather, theseaspects are provided so that this disclosure will be thorough andcomplete, and will fully convey the scope of the aspects to thoseskilled in the art. Aspects may be practiced as methods, systems ordevices. Accordingly, aspects may take the form of a hardwareimplementation, an entirely software implementation or an implementationcombining software and hardware aspects. The following detaileddescription is, therefore, not to be taken in a limiting sense.

The present disclosure provides systems and methods for capturing datato acquire indoor and outdoor texture and geometry. In aspects, a datacapture system may be configured to acquire texture data, geometry data,navigation data and/or orientation data to support geolocation andgeoreferencing in indoor and outdoor environments. Georeferencing, asused herein, may refer to a process of associating a physical map or animage of a map, for example, with spatial locations. The data capturesystem may further be configured to acquire seamless texture data from a360° horizontal and vertical perspective to support panoramic video andimages.

In examples, a data capture system may receive data from a navigationsystem, such as a Global Navigation Satellite System (GNSS). The GNSS,for example, may be configured to provide autonomous geo-spatialpositioning with global coverage and to provide time signals that allowsa receiver to determine its location with precision (e.g., within a fewmeters). The GNSS may be configured to provide such data along a line ofsight to the receiver, such that an obstruction to the line of sightbetween the GNSS and the receiver precludes the transmission of theinformation. In another example, the data capture system may receivedata from a cellular network system. The cellular network system may beconfigured to provide the position of a mobile device using radiosignals emitted by a plurality of radio towers at known locations thatbroadcast signals at regular intervals. The cellular network system mayuse techniques to locate the mobile device, such as triangulation,multilateration and trilateration.

The data capture system may perform a kinematic alignment afterreceiving the data. Kinematic alignment, as used herein, may refer tothe process of computing the GNSS-based position of a receiver andpositioning the receiver such that a GNSS antenna and an inertialmeasurement unit (IMU) associated with the receiver are aligned. Inaspects, performing a kinematic alignment allows the data capture systemto synchronize one or more sensors associated with the data capturesystem. Synchronizing the sensors allows the data capture system toacquire synchronized sensor data. Synchronized sensor data, as usedherein, may refer to data acquired from sensors that have been alignedin time (e.g., via a single source clock) or position with respect toeach other or a common point of reference. Examples of sensors mayinclude Light Detection and Ranging (LIDAR) sensors, optical sensors,acoustical sensors, magnetic sensors, gyroscopic sensors, mechanicalsensors, etc.

The data capture system may record the synchronized sensor data. Thesensor data may comprise images, video, geometry, surface properties,navigational data and/or orientation data of the surrounding environmentand/or the data capture system. In some aspects, the sensor data may beaffixed with the global time retrieved from the GNSS. In other aspects,the GNSS data may not be available and the sensor data may be affixedwith the local time. In aspects, the data capture system may update itsposition at intervals (e.g., continuously, periodically, on-demand,etc.) and may transmit the positional data to one or more computingunits. One skilled in the art will recognize that examples described inthe present disclosure may be applicable to alternate systems ordevices.

FIG. 1 illustrates an overview of an example system 100 for capturingdata to acquire indoor and outdoor texture and geometry as describedherein. Exemplary system 100 presented is a combination ofinterdependent components that interact to form an integrated whole forimproving recommendations from implicit feedback. Components of thesystems may be hardware components or software implemented on and/orexecuted by hardware components of system 100. In examples, system 100may include any of hardware components (e.g., used to execute/runoperating system (OS)), and software components (e.g., applications,application programming interfaces, modules, virtual machines, runtimelibraries, etc.) running on hardware.

In one example, an exemplary system 100 may provide an environment forsoftware components to run, obey constraints set for operating, andmakes use of resources or facilities of the system 100, where componentsmay be software (e.g., application, program, module, etc.) running onone or more processing devices. For instance, software (e.g.,applications, operational instructions, modules, etc.) may be run on aprocessing device such as a computer, mobile device (e.g.,smartphone/phone, tablet) and/or any other electronic devices. As anexample of a processing device operating environment, refer to theexemplary operating environments depicted in FIGS. 5-7. In otherexamples, the components of systems disclosed herein may be spreadacross multiple devices. For instance, input may be entered on aprocessing device and information may be processed or accessed fromother devices in a network such as one or more server devices.

As one example, the system 100 comprises client system 102A, clientsystem 102B, distributed network 104, navigation device 106A, navigationdevice 106B and navigation device 106C. One of skill in the art willappreciate that the scale of systems such as system 100 may vary and mayinclude more or fewer components than those described in FIG. 1. In someexamples, interfacing between components of the system 100 may occurremotely, for example where components of system 100 may be spreadacross one or more devices of a distributed network.

In aspects, client system 102A may be configured to acquire the objectgeometry and object surface properties of the surrounding environment inorder to perform photogrammetry and object reconstruction. In oneexample, client system 102A may receive signal data from one or moresatellites, such as navigation devices 106A, 106B, and 106C. Navigationdevice 106A, for example, may be configured to broadcast or transmitinformation about its position and the current time at regular (orirregular) intervals to one or more receivers. Client system 102A maytimestamp the signal data with a global time of the one or moresatellites, and perform kinematic alignment for one or more componentsof client system 102A. Client system 102A may then record sensor datafrom the surrounding environment, including texture data, geometry data,navigation data and/or orientation data for one or more objects. Theglobal time received from the signal data may be applied to therespective sensor data as it recorded. The sensor data may be providedto a processing component of, or associated with, the client system 102Aas feedback. For example, client system 102A may transmit the sensordata to client system 102B via distributed network 104. The processingcomponent may process the sensor data and provide traveled trajectoryand/or object geometry data to a user interface. In another example,signal data may not be available to client system 102A or the signaldata may be insufficient. In such an example, the recorded sensor datamay not be time-stamped or may be time-stamped with the local time.

In some aspects, client system 102A and client system 102B may beconfigured to simultaneously or alternately receive signal data from theone or more satellites and/or record sensor data from the surroundingenvironment(s). For example, client system 102A and client system 102Bmay be in separate locations (e.g., 20 miles apart). Client system 102Amay be configured to receive signal data from one or more satellites.Client system 102A may parse the signal data to identify global timedata and may transmit the global time data to client system 102B. Clientsystem 102B may acquire record sensor data from the surroundingenvironment and may and may apply the received global time data to thesensor data. The sensor data may then be processed by client system 102Bor transmitted to a separate computing device for processing a displayon a user interface.

FIG. 2A illustrates an overview of a data capture system 200 asdescribed herein. The data capture techniques implemented by datacapture system 200 may comprise the data capture techniques and inputdescribed in FIG. 1. In alternative examples, multiple system(comprising one or more components such as processor and/or memory) mayperform the processing described in systems 100 and 200, respectively.Further, data capture system 200 may comprise a user interface componentas described in the description of FIG. 1.

Exemplary data capture system 200 may comprise panoramic head 202,visual odometry system 208, LIDAR sensor system 206, GNSS antenna 204,orientation and navigation system 210, and/or processing unit(s) 212. Inaspects, one or more these components may be mounted to a rigid frame.In some aspects, the rigid frame and mounted components may beconfigured to be portably maneuvered about an environment. Alternatelyor additionally, the rigid frame may be configured to be positioned on astationary device (e.g., a tripod).

In a particular example, as shown in FIG. 2B, a rigid frame may besubstantially rectangular. The rigid frame may comprise a length andwidth that approximates a backpack or that is suitable for transport viaa human user. The rigid frame may include a rectangular shelf that isattached orthogonally from the rigid frame. The shelf may comprise awidth that is substantially similar to the rigid frame and a length thatis approximately one-third the length of the rigid frame. The shelf maybe positioned at or near the bottom of the rigid frame and may include astand. The stand may by attached orthogonally to the shelf and mayextend from the bottom of the shelf, such that the rigid frame may bewill remain upright when placed on a flat surface. Panoramic head 202may be attached to (and extend from) the top of the rigid frame, suchthe panoramic head is afforded a view that is minimally occluded by theuser and/or the other components of data capture system 200. LIDARsensor system 206 may also be attached to (and extend from) the top ofthe rigid frame, such that the line of sight to objects in theenvironment is unobstructed. GNSS antenna 204 may be attached to aconstruct that is mounted to the rigid frame. The construct may besimilar in length and height to the rigid construct and may beconfigured to securely affix one or more components of data capturesystem 200. GNSS antenna 204 may be positioned near the top of theconstruct to reduce signal interference. Visual odometry system 208 andorientation and navigation system 210 may be attached to the constructbelow GNSS antenna 204. Processing unit(s) 212 may be attached to theconstruct below visual odometry system 208 and orientation andnavigation system 210.

Panoramic head 202 may be configured with a plurality of individualoptical cameras, such that the combined field of view of the individualcameras represents complete (or substantially complete) coverage of thesurrounding environment (e.g., a 360 degree view). In some aspects, asillustrated in FIG. 3A, the individual cameras 302, 304, 306, forexample, may be mounted to a rigid construction 308 such that thephysical distance of the focal points of each individual camera to thefocal point of one or more adjacent cameras is small enough to avoidvisible parallaxes in the panoramic image. In examples, the individualcameras may be configured to acquire input data, such as images, stillvideo, video, infrared light, x-ray radiation, etc.

Visual odometry system 208 may be configured to use input data (e.g.,images, still video, and/or video) to calculate a change in positionover time. Visual odometry, as used herein, may refer to the process ofdetermining odometry information using visual data to estimate thedistance traveled. Visual odometry systems may be used in, for example,robotic applications (e.g., Hans Moravec's work with the Stanford Cart),autonomous flight applications (e.g., the Mars Exploration Rovers) andground vehicle applications (e.g., urban mapping and georeferencing). Inaspects, visual odometry system 208 may acquire input data and establisha correspondence between two or more portions of the input data. In aparticular example, establishing a correspondence comprises defining oneor more interest operators across a plurality of images and matching theinterest operators across two or more images. Interest operators, asused herein, may refer to algorithms which detect features of interestin an image, such as corners, edges, objects, etc. Visual odometrysystem 208 may construct an optical flow field using the establishedcorrespondence. An optical flow field, as used herein, may refer to arepresentation of the apparent motion of objects, surfaces and edges ina visual scene caused by the relative motion between and observer andthe scene. In a particular example, a plurality of images may be used toconstruct an optical flow field that may be analyzed by a visualodometry system for potential tracking errors and outliers. Visualodometry system 208 may then estimate the motion of the camera(s) fromthe optical flow field. Estimates of the motion may be generated usingtechniques such as linear quadratic estimation (LQE) for state estimatedistribution maintenance or random sampling to find the geometric andthree dimensional properties of the features that minimize a costfunction.

LIDAR sensor system 206 may be configured to measure the distancebetween the LIDAR sensor system 206 and one or more objects. LIDARsensor systems may be used, for example, to create high-resolution mapsfor applications in geomatics, archaeology, geography, geology,seismology, remote sensing, atmospheric physics, laser altimetry andcontour mapping. In examples, LIDAR sensor system 206 may fire rapidpulses of laser light at the surface of one or more objects and measuresthe amount of time that each pulse takes to bounce of the objects andreturn to a sensor on LIDAR sensor system 206. The LIDAR sensor data mayenable LIDAR sensor system 206 to measure and map a 3D representation ofthe surrounding environment. For example, the sensor data may be used toconstruct one or more depth images of objects in the surroundingenvironment. A geometry may be constructed to represent the objectsusing the depth images. The geometry may be void of texture data andcolors data, but may include spatial relationship data of the objects inthe environment. In some aspects, LIDAR sensor system 206 may be used tocreate a three dimensional representation of urban environments (e.g.,via street-side data) and/or indoor environments (e.g., shopping malls,airport buildings, a single room, etc.).

GNSS antenna 204 may be configured to receive data from a navigationsystem, such as a GNSS. In examples, GNSS antenna 204 may capture thesignals transmitted by one or more satellites and may convert the energyin the electromagnetic waves, for example, arriving from the satellitesinto an electric current that is further processed by a receiver. Thedata may comprise, for example, ranging signals for measuring thedistance to the satellite. The data may also comprise navigationmessages for calculating the position of each satellite in orbit andrelaying information about the time and status of an entire satelliteconstellation (e.g., the almanac). In some aspects, GNSS antenna 204 maybe mounted on a rigid frame of data capture system 200.

Orientation and navigation system 210 may be configured to acquirereceived data from GNSS antenna 204 and interpolate movement data fordata capture system 200. Orientation and navigation systems may be usedto, for example, maneuver airborne systems (e.g., unmanned aerialvehicles and aircrafts), ground vehicles and devices, and aquaticsystems (e.g., submarines and probes). In aspects, orientation andnavigation system 210 may measure an object's velocity, orientation andgravitational forces using various sensors, such as accelerometers,gyroscopes, and/or magnetometers. In a particular example, orientationand navigation system 210 may measure the position and orientation ofdata capture system 200 as data capture system 200 is moved about anenvironment. Orientation and navigation system 210 may then calculatethe subsequent position of the data capture system 200 using known orestimated speeds over elapsed time and course. In some aspects,orientation and navigation system 210 may be mounted on a rigid frame ofdata capture system 200.

Processing unit(s) 212 may be configured to receive and process datafrom one or more of the other components of input processing unit 200.In aspects, processing unit(s) 212 may synchronize the sensors of eachcomponent of input processing unit 200; receive and stitch togetherinput data from panoramic head 202; process the input operators andstore the optical flow field of visual odometry system 208; create athree dimensional representation of an environment using the sensor dataof LIDAR sensor system 206 and/or the movement data of orientation andnavigation system 210; perform various techniques for image correction(e.g., rolling shutter effect compensation); and/or provide the threedimensional representation (and/or other information) to a userinterface.

In aspects, data capture system 200 may be configured to provideocclusion management for seamless panoramic coverage. In examples, theindividual cameras mounted on panoramic head 202 may be configured in anarrangement such that those portions of an individual camera field ofview which are occluded are captured by at least one of the otherindividual cameras. In a particular example, the physical distance ofthe focal points of each camera to the focal points of the adjacentcamera is approximately 60 mm. The input data acquired by panoramic head202 may then be stitched together by software executing on processingunit(s) 212, for example, to create a seamless panoramic image or video.Stitching, as used herein, may refer to combining multiple,partially-overlapping images of a scene to create a single,uninterrupted image of the scene.

In some aspects, data capture system 200 may be further or alternatelyconfigured to provide for synchronization of various components of theinput processing unit 200. In examples, one or more sensors from one ormore components of the input processing unit 200 may be synchronized byprocessing unit(s) 212. The synchronization may be such that thegeometry reconstruction results (e.g., the three dimensional mapping by,for example, LIDAR sensor system 206) and the positional recognition ofeach sensor may be transformed into a combined spatial coordinatesystem. The six parameters of such transformation (e.g. threedimensional translation and rotation) are carefully determined from acalibration procedure for one or more components of the system anddescribe in such way pose and position of the individual component(e.g., the sensor associated with the component) with respect to therigid frame of the data capture system. The combined spatial coordinatesystem may be constructed such that the accuracy of the synchronizedsensors is maintained even if data capture system 200 is in motionduring data acquisition. In some examples, the synchronization of thesensors may be managed via a quartz clock that is coupled to, oraccessible by, processing unit(s) 212. The quartz clock may provide for,for example, frame by frame synchronization of video captured by one ormore individual cameras of panoramic head 202.

In aspects, data capture system 200 may be further or alternatelyconfigured with an active/passive cooling system. In examples, theindividual cameras of panoramic head 202 may gradually increase in heatduring operation. To combat overheating of the individual cameras,panoramic head 202 comprises an active cooling system and a passivecooling system. FIG. 3B illustrates an exemplary active and passivecooling system of panoramic head 202. The cooling system is designed insuch way that an air stream is guided through the panoramic sensor headin order to transport the caloric energy produced from the one ormultiple optical camera heads and the active part of the cooling systemboosts the air stream if the natural convection is not efficient. InFIG. 3B, the active part of the cooling system is situated in the middleportion of the panoramic head and includes a fan assembly and a flowchannel, as indicated by the arrow 316. In some examples, the fanassembly may be configured to provide cooled air to flow channel. Flowchannel may be configured to receive and carry the cooled air to one ormore individual cameras, to portions of the panoramic head, and/or toother components of a data capture system. Continuing with FIG. 3B, thepassive part of the cooling system indicated by the arrow 318incorporates a design of the panoramic head that supports dispersing thecooled air provided by the fan assembly and cooling the individualcameras. In some examples, the design may comprise equipping one or moreindividual cameras with one or more cooling fins. The cooling fins maybe configured to enhance the cooling effect of the cooled air to the oneor more individual cameras.

In aspects, data capture system 200 may be further or alternatelyconfigured with a flexible software readout mechanism. The flexiblesoftware readout mechanism may allow a user to use configurable softwaresettings to select and alter the read out direction for each camera andthus, the behavior of the rolling shutter effects. In examples, one ormore individual cameras of data capture system 200 may use rollingshutter. Rolling shutter, as used herein, may refer to a method of imagecapture in which a still picture or each frame of a video is captured byrapidly scanning a scene, instead of taking a snapshot of the entirescene at a single instant in time. In some examples, the imagesgenerated using rolling shutter may be distorted when the one or moreindividual cameras are moving relative to the object being captured. Theeffect may worsen when the images are stitched together and the effectof each image is seen to point in different directions. In suchexamples, the flexible software readout mechanism may allow for aselection of the readout direction for each individual camera sensor.This selection may allow the images to be altered such that the imagedistortion for each image is at the same magnitude and direction asthose of the adjacent image; thus providing an accurately stitchedtogether image.

Accordingly, the present disclosure provides a plurality of technicalbenefits including but not limited to: panoramic video and imagecapture; occlusion management; active and passive cooling techniques;video input device synchronization, rolling shutter effectscompensation; data capture time and resource cost reduction; andimproved image quality, among other examples.

FIG. 4 illustrates a process flow associated with a data capture systemfor indoor and outdoor texture and geometry as described herein. Inaspects, method 400 may be executed by an exemplary system such assystem 100 of FIG. 1. In examples, method 400 may be executed on adevice comprising at least one processor configured to store and executeoperations, programs or instructions. However, method 400 is not limitedto such examples. In other examples, method 400 may be performed on anapplication or service for processing video and/or images. In at leastone example, method 400 may be executed (e.g., computer-implementedoperations) by one or more components of a distributed network, forinstance, web service/distributed network service (e.g. cloud service)to leverage signal data analysis, graphical processing and statisticalanalysis.

Exemplary method 400 begins at operation 402 where signal data isreceived by a data capture system. The signal data may be received froma navigation system, such as GNSS. The GNSS may be configured to providesuch data along a line of sight to the receiver, such that anobstruction to the line of sight between the GNSS and the receiverprecludes the transmission of the information. The signal data maycomprise geo-spatial positioning information and time signals that allowa receiver to accurately determine the location of the GNSS.

At operation 404, the signal data may be processed by the data capturesystem. In examples, processing the signal data may include parsing thesignal data to identify various pieces of information, such asgeo-spatial positioning information and global time signals. In oneparticular example, the geo-spatial positioning information may betransmitted to a storage location and tracked over time. The global timesignals may be used create one or more timestamps. The timestamps may betransmitted to one or more components of the data capture system to pairwith, and/or affix to, images, video, sill video and/or other sensordata.

At operation 406, the data capture system may perform a kinematicalignment after receiving the signal data. In aspects, one or morecomponents of the data capture system may be mounted to a portable rigidframe. The position of one or more sensors of the one or more componentsmay be calibrated in pose and position with respect to the rigid frame.In examples, this calibration (e.g., kinematic alignment) provides eachsensor coordinate system with, for example, a three dimensionalorientation by measuring gravity and the movement direction of thesensor. The calibration also allows the use of timestamps to synchronizeeach sensor (or sensor system) to a common clock associated with thedata capture system and a spatial coordinate system with position andorientation of the data capture system.

At operation 408, the data capture system may record environmental data.In aspects, one or more components of the data capture system maysimultaneously or alternately acquire data from the surroundingenvironment. For example, a LIDAR sensor may measure and record thedistance between the LIDAR sensor and one or more objects in theenvironment at intervals. The LIDAR sensor data may be used to measureand map a three dimensional representation of the surroundingenvironment. As another example, a panoramic head may record imageimages, still video, and/or video of an environment. The image data maybe used to provide color and texture to geometry created by, forexample, the LIDAR sensor. As another example, a visual odometry systemmay record the change in position over time of one or more sensors ofthe data capture system. The change in position data may be used toestablish a correspondence between two or more portions of the inputdata. In yet another example, an orientation and navigation system mayrecord the velocity, orientation and gravitational forces on one or moreobjects in the environment. This data may be used to interpolatemovement data within the image data acquired and generated by the datacapture system.

In aspects, the recorded environmental data may be affixed withtimestamps derived from the global time received from the GNSS. In otheraspects, the GNSS signal data may not be available because, for example,the line of sight between the GNSS and the receiver is obstructed. Insuch aspects, timestamps may be generated using the local time of acommon clock associated with the data capture system. The recordedenvironmental data may be affixed with timestamps derived from thecommon clock. In a particular example, the common clock is a quartzclock by which all of the sensors of the data capture system aremanaged. The timestamps may allow the data capture system to accuratelyidentify the sequence in which the environmental data (e.g., images,video, etc.) was recorded. The data capture system may then use thetimestamps to pair the environmental data with data from other sensorsof the data capture system.

At operation 410, the data capture system may provide feedback to a userinterface. In examples, the feedback may comprise various pieces ofinformation, such as a colored and textured three dimensionalrepresentation of the surrounding environment, a video rendering, sensordata from one or more components of the data capture system, signal datafrom the GNSS, data and graphics analysis information, presentationoptions, etc. In some examples, the feedback may be viewed on a userinterface that is communicatively coupled to the data capture system. Inother examples, the feedback may be transmitted to a remote computingdevice having access to a user interface.

FIGS. 5-7 and the associated descriptions provide a discussion of avariety of operating environments in which examples of the invention maybe practiced. However, the devices and systems illustrated and discussedwith respect to FIGS. 5-7 are for purposes of example and illustrationand are not limiting of a vast number of computing device configurationsthat may be utilized for practicing examples of the invention, describedherein.

FIG. 5 is a block diagram illustrating physical components of acomputing device 502, for example a component of a system with whichexamples of the present disclosure may be practiced. The computingdevice components described below may be suitable for the computingdevices described above. In a basic configuration, the computing device502 may include at least one processing unit 504 and a system memory506. Depending on the configuration and type of computing device, thesystem memory 506 may comprise, but is not limited to, volatile storage(e.g., random access memory), non-volatile storage (e.g., read-onlymemory), flash memory, or any combination of such memories. The systemmemory 506 may include an operating system 507 and one or more programmodules 508 suitable for running software applications 520 such asapplication 528, IO manager 524, and other utility 526. As examples,system memory 506 may store instructions for execution. Other examplesof system memory 506 may be components such as a knowledge resource orlearned program pool, as examples. The operating system 507, forexample, may be suitable for controlling the operation of the computingdevice 502. Furthermore, examples of the invention may be practiced inconjunction with a graphics library, other operating systems, or anyother application program and is not limited to any particularapplication or system. This basic configuration is illustrated in FIG. 5by those components within a dashed line 522. The computing device 502may have additional features or functionality. For example, thecomputing device 502 may also include additional data storage devices(removable and/or non-removable) such as, for example, magnetic disks,optical disks, or tape. Such additional storage is illustrated in FIG. 5by a removable storage device 509 and a non-removable storage device510.

As stated above, a number of program modules and data files may bestored in the system memory 506. While executing on the processing unit504, the program modules 508 (e.g., application 528, Input/Output (I/O)manager 524, and other utility 526) may perform processes including, butnot limited to, one or more of the stages of the operational method 400illustrated in FIG. 4, for example. Other program modules that may beused in accordance with examples of the present invention may includeelectronic mail and contacts applications, word processing applications,spreadsheet applications, database applications, slide presentationapplications, input recognition applications, drawing or computer-aidedapplication programs, etc.

Furthermore, examples of the invention may be practiced in an electricalcircuit comprising discrete electronic elements, packaged or integratedelectronic chips containing logic gates, a circuit utilizing amicroprocessor, or on a single chip containing electronic elements ormicroprocessors. For example, examples of the invention may be practicedvia a system-on-a-chip (SOC) where each or many of the componentsillustrated in FIG. 5 may be integrated onto a single integratedcircuit. Such an SOC device may include one or more processing units,graphics units, communications units, system virtualization units andvarious application functionality all of which are integrated (or“burned”) onto the chip substrate as a single integrated circuit. Whenoperating via an SOC, the functionality described herein may be operatedvia application-specific logic integrated with other components of thecomputing device 502 on the single integrated circuit (chip). Examplesof the present disclosure may also be practiced using other technologiescapable of performing logical operations such as, for example, AND, OR,and NOT, including but not limited to mechanical, optical, fluidic, andquantum technologies. In addition, examples of the invention may bepracticed within a general purpose computer or in any other circuits orsystems.

The computing device 502 may also have one or more input device(s) 512such as a keyboard, a mouse, a pen, a sound input device, a device forvoice input/recognition, a touch input device, etc. The output device(s)514 such as a display, speakers, a printer, etc. may also be included.The aforementioned devices are examples and others may be used. Thecomputing device 504 may include one or more communication connections516 allowing communications with other computing devices 518. Examplesof suitable communication connections 516 include, but are not limitedto, RF transmitter, receiver, and/or transceiver circuitry; universalserial bus (USB), parallel, and/or serial ports.

The term computer readable media as used herein may include computerstorage media. Computer storage media may include volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information, such as computer readableinstructions, data structures, or program modules. The system memory506, the removable storage device 509, and the non-removable storagedevice 510 are all computer storage media examples (i.e., memorystorage.) Computer storage media may include RAM, ROM, electricallyerasable read-only memory (EEPROM), flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other article of manufacturewhich can be used to store information and which can be accessed by thecomputing device 502. Any such computer storage media may be part of thecomputing device 502. Computer storage media does not include a carrierwave or other propagated or modulated data signal.

Communication media may be embodied by computer readable instructions,data structures, program modules, or other data in a modulated datasignal, such as a carrier wave or other transport mechanism, andincludes any information delivery media. The term “modulated datasignal” may describe a signal that has one or more characteristics setor changed in such a manner as to encode information in the signal. Byway of example, and not limitation, communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, radio frequency (RF), infrared, andother wireless media.

FIGS. 6A and 6B illustrate a mobile computing device 600, for example, amobile telephone, a smart phone, a personal data assistant, a tabletpersonal computer, a laptop computer, and the like, with which examplesof the invention may be practiced. For example, mobile computing device600 may be implemented as system 100, components of systems 100 may beconfigured to execute processing methods as described in FIG. 4, amongother examples. With reference to FIG. 6A, one example of a mobilecomputing device 600 for implementing the examples is illustrated. In abasic configuration, the mobile computing device 600 is a handheldcomputer having both input elements and output elements. The mobilecomputing device 600 typically includes a display 605 and one or moreinput buttons 610 that allow the user to enter information into themobile computing device 600. The display 605 of the mobile computingdevice 600 may also function as an input device (e.g., a touch screendisplay). If included, an optional side input element 615 allows furtheruser input. The side input element 615 may be a rotary switch, a button,or any other type of manual input element. In alternative examples,mobile computing device 600 may incorporate more or less input elements.For example, the display 605 may not be a touch screen in some examples.In yet another alternative example, the mobile computing device 600 is aportable phone system, such as a cellular phone. The mobile computingdevice 600 may also include an optional keypad 635. Optional keypad 635may be a physical keypad or a “soft” keypad generated on the touchscreen display. In various examples, the output elements include thedisplay 605 for showing a graphical user interface (GUI), a visualindicator 620 (e.g., a light emitting diode), and/or an audio transducer625 (e.g., a speaker). In some examples, the mobile computing device 600incorporates a vibration transducer for providing the user with tactilefeedback. In yet another example, the mobile computing device 600incorporates input and/or output ports, such as an audio input (e.g., amicrophone jack), an audio output (e.g., a headphone jack), and a videooutput (e.g., a HDMI port) for sending signals to or receiving signalsfrom an external device.

FIG. 6B is a block diagram illustrating the architecture of one exampleof a mobile computing device. That is, the mobile computing device 600can incorporate a system (i.e., an architecture) 602 to implement someexamples. In examples, the system 602 is implemented as a “smart phone”capable of running one or more applications (e.g., browser, e-mail,input processing, calendaring, contact managers, messaging clients,games, and media clients/players). In some examples, the system 602 isintegrated as a computing device, such as an integrated personal digitalassistant (PDA) and wireless phone.

One or more application programs 666 may be loaded into the memory 662and run on or in association with the operating system 664. Examples ofthe application programs include phone dialer programs, e-mail programs,personal information management (PIM) programs, word processingprograms, spreadsheet programs, Internet browser programs, messagingprograms, and so forth. The system 602 also includes a non-volatilestorage area 668 within the memory 662. The non-volatile storage area668 may be used to store persistent information that should not be lostif the system 602 is powered down. The application programs 666 may useand store information in the non-volatile storage area 668, such ase-mail or other messages used by an e-mail application, and the like. Asynchronization application (not shown) also resides on the system 602and is programmed to interact with a corresponding synchronizationapplication resident on a host computer to keep the information storedin the non-volatile storage area 668 synchronized with correspondinginformation stored at the host computer. As should be appreciated, otherapplications may be loaded into the memory 662 and run on the mobilecomputing device 600, including application 528, IO manager 524, andother utility 526 described herein.

The system 602 has a power supply 670, which may be implemented as oneor more batteries. The power supply 670 might further include anexternal power source, such as an AC adapter or a powered docking cradlethat supplements or recharges the batteries.

The system 602 may include peripheral device port 630 that performs thefunction of facilitating connectivity between system 602 and one or moreperipheral devices. Transmissions to and from the peripheral device port630 are conducted under control of the operating system 664. In otherwords, communications received by the peripheral device port 630 may bedisseminated to the application programs 666 via the operating system664, and vice versa.

The system 602 may also include a radio 672 that performs the functionof transmitting and receiving radio frequency communications. The radio672 facilitates wireless connectivity between the system 602 and the“outside world,” via a communications carrier or service provider.Transmissions to and from the radio 672 are conducted under control ofthe operating system 664. In other words, communications received by theradio 672 may be disseminated to the application programs 666 via theoperating system 664, and vice versa.

The visual indicator 620 may be used to provide visual notifications,and/or an audio interface 674 may be used for producing audiblenotifications via the audio transducer 625. In the illustrated example,the visual indicator 620 is a light emitting diode (LED) and the audiotransducer 625 is a speaker. These devices may be directly coupled tothe power supply 670 so that when activated, they remain on for aduration dictated by the notification mechanism even though theprocessor 660 and other components might shut down for conservingbattery power. The LED may be programmed to remain on indefinitely untilthe user takes action to indicate the powered-on status of the device.The audio interface 674 is used to provide audible signals to andreceive audible signals from the user. For example, in addition to beingcoupled to the audio transducer 625, the audio interface 674 may also becoupled to a microphone to receive audible input, such as to facilitatea telephone conversation. In accordance with examples of the presentinvention, the microphone may also serve as an audio sensor tofacilitate control of notifications, as will be described below. Thesystem 602 may further include a video interface 676 that enables anoperation of an on-board camera 630 to record still images, videostream, and the like.

A mobile computing device 600 implementing the system 602 may haveadditional features or functionality. For example, the mobile computingdevice 600 may also include additional data storage devices (removableand/or non-removable) such as, magnetic disks, optical disks, or tape.Such additional storage is illustrated in FIG. 6B by the non-volatilestorage area 668.

Data/information generated or captured by the mobile computing device600 and stored via the system 602 may be stored locally on the mobilecomputing device 600, as described above, or the data may be stored onany number of storage media that may be accessed by the device via theradio 672 or via a wired connection between the mobile computing device600 and a separate computing device associated with the mobile computingdevice 600, for example, a server computer in a distributed computingnetwork, such as the Internet. As should be appreciated suchdata/information may be accessed via the mobile computing device 600 viathe radio 672 or via a distributed computing network. Similarly, suchdata/information may be readily transferred between computing devicesfor storage and use according to well-known data/information transferand storage means, including electronic mail and collaborativedata/information sharing systems.

FIG. 7 illustrates one example of the architecture of a system forproviding an application that reliably accesses target data on a storagesystem and handles communication failures to one or more client devices,as described above. Target data accessed, interacted with, or edited inassociation with application 528, IO manager 524, other utility 526, andstorage may be stored in different communication channels or otherstorage types. For example, various documents may be stored using adirectory service 722, a web portal 724, a mailbox service 726, aninstant messaging store 728, or a social networking site 730,application 528, IO manager 524, other utility 526, and storage systemsmay use any of these types of systems or the like for enabling datautilization, as described herein. A server 720 may provide storagesystem for use by a client operating on general computing device 502 andmobile device(s) 600 through network 715. By way of example, network 715may comprise the Internet or any other type of local or wide areanetwork, and client nodes may be implemented as a computing device 502embodied in a personal computer, a tablet computing device, and/or by amobile computing device 600 (e.g., a smart phone). Any of these examplesof the client computing device 502 or 600 may obtain content from thestore 716.

Reference has been made throughout this specification to “one example”or “an example,” meaning that a particular described feature, structure,or characteristic is included in at least one example. Thus, usage ofsuch phrases may refer to more than just one example. Furthermore, thedescribed features, structures, or characteristics may be combined inany suitable manner in one or more examples.

One skilled in the relevant art may recognize, however, that theexamples may be practiced without one or more of the specific details,or with other methods, resources, materials, etc. In other instances,well known structures, resources, or operations have not been shown ordescribed in detail merely to observe obscuring aspects of the examples.

While sample examples and applications have been illustrated anddescribed, it is to be understood that the examples are not limited tothe precise configuration and resources described above. Variousmodifications, changes, and variations apparent to those skilled in theart may be made in the arrangement, operation, and details of themethods and systems disclosed herein without departing from the scope ofthe claimed examples.

What is claimed is:
 1. A system comprising: at least one processor; andmemory coupled to the at least one processor, the memory comprisingcomputer executable instructions that, when executed by the at least oneprocessor, performs a method comprising: receiving signal data;processing the signal data, wherein the processing comprises determininga time associated with the signal data; synchronizing one or moresensors of the system; using the one or more sensors, acquiringenvironmental data from an environment; and providing feedbackassociated with the acquired environmental data.
 2. The system of claim1, wherein the signal data is received from a global navigation system.3. The system of claim 1, wherein the system comprises a rigid frame,wherein the one or more sensors are mounted to the rigid frame.
 4. Thesystem of claim 3, wherein the synchronizing comprises aligning the oneor more sensors in pose and position with respect to the rigid frame. 5.The system of claim 1, wherein the acquiring comprises at least one of:measuring a distance between at least one of the one or more sensors andone or more objects in the environment; recording image data of theenvironment, wherein the image data comprises at least one of the groupconsisting of images, still video and video; recording a change inposition over time of the one or more sensors; and measuring a velocity,an orientation, and gravitational forces of one or more objects in theenvironment.
 6. The system of claim 1, wherein the environmental data isused to construct a geometry representing the environment, and whereinthe environmental data is further used to texture the geometry.
 7. Thesystem of claim 6, wherein the environmental data is further used tointerpolate movement data within the textured geometry.
 8. The system ofclaim 1, wherein the feedback comprises a three-dimensional, texturedrepresentation of the environment.
 9. The system of claim 1, furthercomprising: generating one or more timestamps using the time associatedwith the signal data, wherein the one or more timestamps are affixed tothe environmental data.
 10. A system comprising: an antenna configuredto receive signal data; a panoramic head configured to acquire imagedata of an environment, wherein the image data comprises at least one ofthe group consisting of images, still video and video; a radar sensorconfigured to measure a distance between the radar sensor and one ormore objects in an environment; a visual odometry sensor configured todetermine a change in a position of the visual odometry sensor overtime; an orientation and navigation sensor configured to determine aposition, a velocity and an orientation of the orientation andnavigation sensor; and a processing unit configured to processinformation received from at least one of the antenna, the panoramichead, the radar sensor, the visual odometry sensor, and the orientationand navigation sensor.
 11. The system of claim 10, further comprising: arigid frame, the rigid frame comprising the antenna, the panoramic head,the radar sensor, and the orientation and navigation sensor.
 12. Thesystem of claim 10, wherein the panoramic head comprises a rigidconstruction and a plurality of cameras, wherein the plurality ofcameras are mounted to the rigid construction in a configuration thatprovides for a panoramic image of the environment.
 13. The system ofclaim 12, wherein the plurality of cameras comprises a first camerahaving a first focal point and a second camera having a second focalpoint, wherein the first camera and the second camera are positionedwith respect to one another to avoid visible parallaxes.
 14. The systemof claim 12, wherein the panoramic head comprises a cooling systemhaving an active portion and a passive portion, wherein the activeportion includes a fan assembly and the passive portion includesaffixing cooling fins to one or more of the plurality of cameras. 15.The system of claim 12, further comprising: a software readout mechanismthat provides for a selection of a readout direction for each of theplurality of cameras.
 16. The system of claim 10, wherein processing theinformation comprises: constructing a geometry representing theenvironment, wherein the geometry is constructed based at least in parton the information; and applying a texture to the geometry.
 17. Thesystem of claim 16, wherein processing the information furthercomprises: identifying a correspondence between two or more portions ofthe information; and generating a representation of an apparent motionof one or more objects in the textured geometry.
 18. The system of claim10, further comprising a system clock, wherein at least the radarsensor, the visual odometry sensor, and the orientation and navigationsensor are synchronized to the system clock.
 19. The system of claim 18,wherein the synchronized sensors are used to transform the informationinto a spatial coordinate system.
 20. A method for capturing data, themethod comprising: receiving, by a data capture system, signal data froma global navigation system; processing the signal data, wherein theprocessing comprises determining a time associated with the signal data;synchronizing one or more sensors of the data capture system, whereinthe synchronizing comprises aligning the one or more sensors in pose andposition with respect to a rigid frame; using the one or more sensors,acquiring environmental data from an environment, wherein theenvironmental data is used to construct a geometry representing theenvironment; and providing feedback associated with the acquiredenvironmental data, wherein the feedback comprises a three-dimensionalrepresentation of the environment.